Spoken dialogue translation systems evaluation: results, new trends, problems and proposals
نویسندگان
چکیده
It is important to evaluate Spoken Dialogue Translation Systems, but as we show by analyzing evaluation methods in the Verbmobil, C-STAR II, and the Nespole! projects, the current state of the art is not fully satisfactory. Subjective methods are too costly, and objective methods, although cheaper, don’t give good indications about usability. We propose some ideas to improve that situation.
منابع مشابه
Evaluating Interactive Dialogue Systems: Extending Component Evaluation to Integrated System Evaluation
This paper discusses the range of ways in which spoken dialogue system components have been evaluated and discusses approaches to evaluation that attempt to integrate component evaluation into an overall view of system performance. We will argue that the PARADISE (PARAdigm for Dialogue System Evaluation) framework has several advantages over other proposals.
متن کاملTask Complexity measurement in the evaluation of Spoken Dialogue Systems
Spoken Dialogue Systems (SDS ́s) is a technology that provides users with a speech based interaction with machines to carry out several activities in the pursuit of their goals (information retrieval, business transactions, automatic translation, etc.). Development of this kind of systems needs an evaluation process to show the real system performance, to identify implementation errors, etc. The...
متن کاملPredicting and Adapting to Poor Speech Recognition in a Spoken Dialogue System
Spoken dialogue system performance can vary widely for different users, as well for the same user during different dialogues. This paper presents the design and evaluation of an adaptive version of TOOT, a spoken dialogue system for retrieving online train schedules. Adaptive TOOT predicts whether a user is having speech recognition problems as a particular dialogue progresses, and automaticall...
متن کاملDesigning a Task-Based Evaluation Methodology for a Spoken Machine Translation System
In this paper, I discuss issues pertinent to the design of a task-based evaluation methodology for a spoken machine translation (MT) system processing human to human communication rather than human to machine communication. I claim that system mediated human to human communication requires new evaluation criteria and metrics based on goal complexity and the speaker's prioritization of goals. 1 ...
متن کاملThe Role of Semantics in Spoken Dialogue Translation Systems
In this paper, we consider the role of semantics in the spoken dialogue translation systems. We begin by looking at some of the key properties of an existing spoken dialogue system, namely the sundial system which provides ight and train information over the telephone, and how these properties aaect the design methodology and functionality of spoken translation systems. These properties include...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2004